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PREFACE 


This  research  was  conducted  under  the  United  States  Air  Force  Summer 
Faculty/Graduate  Student  Research  Program  and  was  sponsored  by  the  Air  Force  Office 
of  Scientific  Research,  Bolling  Air  Force  Base,  Washington,  D.C.  under  contract  number 
F49620-90-C-0076.  This  research  was  conducted  by  the  Department  of  Communication 
Design,  California  State  University,  Chico  on  behalf  of  the  U.S.  Air  Force  Armstrong 
Laboratory,  Human  Resources  Directorate,  Brooks  Air  Force  Base.  The  Technical 
Training  Research  Division,  Instractional  Design  Branch  (AL/HRTC)  provided 
management  oversight  for  this  effort. 


SUMMARY 


Computer-based  media  production  tools  have  matured  sufficiently  to  enable  the  Air 
Force  to- readily  provide  very  powerful  curriculum  development  tools  based  on  the 
existing  Desktop  IH  or  IV.  However,  providing  instructional  designers  and  developers 
with  a  multimedia  development  workstation  is  not  equivalent  to  providing  them  the 
power  to  use  them  well.  While  an  Automated  Instructional  Design  Advisor  (AIDA)  will 
certainly  aid  designers  and  developers  in  choosing  appropriate  media  to  solve  instruction 
problems,  provision  of  such  powerful  media  production  tools  will  require  a  commitment 
within  the  Air  Force  to  provide  technical  and  creative  support.  Only  this  will  ensure 
effective,  motivational  media  design. 

An  examination  of  matured  computer-based  media  production  technology  was 
undertaken  and  a  group  of  ISD  experts  was  impaneled  to  discern  which  available  tools 
hold  the  most  promise  and  value  for  instructional  design.  This  study  presents  the  finding 
of  the  Delphi  panel  as  well  as  considers  the  impact  of  providing  such  tools  to  designers 
and  developers.  We  make  the  recommendation  that  implementation  of  a  computer- 
mediated  communication  system  be  concurrent  with  the  emplacement  of  computer-based 
media  production  tools  to  create  a  collaborative  instructional  development  environment 
that  will  improve  media  creativity  and  dynamism  especially  with  respect  to  computer- 
based  training.  In  addition,  such  a  system  will  provide  for  centralized  archiving  of 
reusable  and  repurposed  media,  effective  formative  and  summative  evaluation,  increased 
collaboration  between  instructional  designers,  developers,  subject-matter  experts,  and 
media  production  experts.  Such  a  system  should  increase  instructional  quality,  employee 
productivity  and  job  satisfaction. 
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Collaborative  IhstrucCkmal  Devdopment  Envirminent:  A  Sta^  for  the  AIDA 
INTRODUCTION 

The  Air  Force  has  identified  a  goal  concomitant  with  the  development  of  the  Advanced 
Instructional  Design  Advisor  (AIDA)  to  create  a  multi-use  instructional  design  workstation  that 
will  provide  designers/developers  the  power  to  locally  produce  instructional  materials  ranging 
from  graphics  to  video  to  computer-based  interactive  training  (Spector,  1990).  The  Collaborative 
Instructional  Development  Environment  (CIDE)  Workstation  is  a  set  of  functional  specifications 
for  hardware,  software  and  network  communications  to  operate  on  the  Air  Force  Desktop  EH  (PC 
80386)  and  FV  (PC  80486).  The  specifications  were  developed  from  the  responses  of  a  Delphi 
committee  asked  to  evaluate  potential  components  of  such  a  system  in  terms  of  their  most 
common  tasks  and  development  efforts.  The  purpose  of  this  research  is  to  ascertain  not  only  the 
computer-based  media  development  tools  required  for  effective  instructional  design  by  the  Air 
Force,  but  to  explore  the  type  of  collaborative  support  environment  that  will  make  available  to 
instructional  developers  the  expertise  necessary  to  produce  curriculum  materials  that  fully  exploit 
the  power  of  media  to  motivate  as  well  as  teach. 

DISCUSSION  OF  THE  PROBLEM 

Well-developed,  appropriate  media  enhances  instructional  quality  (Johnston,  1987). 

However,  its  use  and  effectiveness  are  hampered  by: 

•  High  cost 

•  Inability  to  determine  appropriate  media 

•  Lack  of  production  expertise 

•  Substantial  lead  time  for  production 

•  Communications  problems 

While  an  Advanced  Instructional  Design  Advisor  (AIDA)  can  aid  in  selecting  the  appropriate 
media  for  an  instructional  activity,  the  problems  listed  represent  serious  impediments  to  the 
instructional  developer  in  incorporating  that  media  into  the  lesson.  An  experienced  live 
instructor  can  often  compensate  for  problems  with  instructional  media,  but  the  trend  toward 
electronic  delivery  of  instruction  demands  highly  effective  stand-alone  media  that  communicates 
and  motivates  (Winn,  1987).  Thus,  there  is  a  pressing  need  to  empower  instructional 
designers/developers  (IDDs)  with  the  tools  to  create  powerful  and  dynamic  instructional  aids 
both  for  standup  instruction  and  for  computer-based  instruction  and  distance  learning. 

The  software  and  hardware  tools  ilow  exist  to  create  a  multimedia  production  workstation 
that  will  allow  development  of  a  wide  array  of  instructional  media  from  documents  to  animation 
to  interactive  digital  video.  However,  it  is  unrealistic  to  believe  that  instructional  developers, 
often  subject  matter  experts  in  a  particular  area  of  training,  will  also  have  the  skills  to  make  the 
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best  use  of  the  powerful  tools  such  a  workstation  will  provide.  Therefore,  how  to  empower 
IDD’s  to  produce  and  manipulate  a  variety  of  instructional  media  without  placing  on  them  the 
burden  of  requisite  expertise  in  what  are  typically  specialist  areas  has  become  a  primary  concern. 
For  example^  providing  a  graphics  software  package  that  is  easy  to  use  does  not  endow  the  user 
with  the  talent  to  generate  visuals  that  are  creative  and  powerful  in  their  ability  to  facilitate  the 
transfer  of  knowledge  and  also  grab  the  learner’s  attention  and  hold  his  or  her  interest. 

AIDA  offers  a  partial  solution.  AIDA  can  assist  an  IDD  in  selecting  an  appropriate  medium 
to  accomplish  specific  instructional  objectives.  It  will  guide  developers  through  many  complex 
design  decisions  and  help  them  clarify  goals  and  directions.  But  AIDA  will  not  be  able  to  review 
the  aesthetics  of  media  development.  It  will  not  be  able  to  examine  a  proposed  media  solution 
for  clarity  of  purpose  and  execution..  Though  it  will  be  able  to  point  out  potential  errors,  it  will 
not  be  able  to  answer  specific  questions  regarding  the  easiest  way  to  solve  a  particular 
communication  problem;  nor  can  AIDA  provide  years  of  media  production  experience  to  a 
neophyte  designer. 

So  while  AIDA  provides  less  experienced  IDD’s  important  guidance  and  assistance  in 
instructional  design  decisions,  it  cannot  give  comprehensive  direction  and  evaluation  in  media 
production..  They  need  both  the  valuable  guidance  of  ADDA  and  a  support  infrastructure  for 
technical  and  creative  assistance  in  making  the  best  use  of  the  increasingly  complex  tools  at  their 
disposal  and  for  producing  the  media  that  will  most  effectively  support  the  instructional 
objectives. 

THE  COLLABORATIVE  INSTRUCTIONAL  DEVELOPMENT  ENVIRONMENT  tCIDEl 

The  increasing  demand  for  multimedia  instructional  materials  has  created  the  need  for  a 
collaborative  instructional  development  environment  that  includes  not  only  instructional  design 
and  subject  matter  expertise,  but  media  development  specialists  as  well.  The  addition  of  a 
communications  network  to  the  computer  platform  on  which  designers/developers  will  be 
running  both  AIDA  and  the  designer-specified  hardware  and  software  for  media  production  can 
create  a  collaborative  open  network  for  sharing  expertise  in  mentor/apprentice  relationships 
between  developers  and  special  interest  advice  conferences  led  by  media  specialists.  Providing 
on-demand  assistance  should  solve  most  media  production  problems  and  provide  a  logical  path 
for  the  instructional  developer  to  travel  from  the  recommendations  of  the  AIDA  to  the  finished 
lesson  material  required  by  the  instructional  design. 

The  inherent  versatility  of  an  open  network  brings  a  number  of  added  benefits  to  Air  Force 
instructional  design  organizations  including: 

•  Centralized  Multimedia  Education  and  Training  Archive  (META)  of  reusable 
instructional  media  for  easy  updates,  adaptation  and  integration. 
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•  Ability  to  search  local  and  remote  archives  for  appropriate  existing  media. 

•  Organizational  gateway  for  send/receive  faxing  and  E-mail. 

•  Centralized  electronic  media  publishing. 

•  Automated  document  flow  for  formative  and  summative  evaluation. 

•  Improved  task  management  and  group  coordination. 

This  study  will  address  the  following  questions  pertinent  to  a  collaborative  instructional 
development  environment: 

1.  Is  electronic  collaboration  effective? 

2.  What  are  the  instructional  media  needs  of  Air  Force  instructional  developers  and  how 
are  they  currently  being  met? 

3.  Can  a  collaborative  instructional  development  environment  be  created  with  current 
commercial-off-the-shelf  technology? 

ASSUMPTIONS 

The  first  assumption  is  that  electronic  media  provide  effective  enhancement  to  instructional 
strategies  in  terms  of  improved  cognition,  information  organization  and  integration,  and  learner 
motivation  has  been  generally  accepted  since  the  1968  research  of  Chu  and  Schramm  (1979). 
Therefore,  this  study  does  not  address  the  use  of  electronic  media  in  improving  the  effectiveness 
of  instruction.  The  term  "effective"  implies  a  judgment  relative  to  a  standard.  This  has 
customarily  meant  comparison  of  some  electronic  medium  relative  to  face-to-face  instruction 
without  electronic  media.  "Medium"  is  defined  by  the  American  Heritage  Dictionary  as  an 
agency,  such  as  a  person,  object  or  quality,  by  means  of  which  something  is  accomplished, 
conveyed  or  transferred.  The  electronic  media  are  simply  carriers  of  information  for  presentation 
to  the  learner.  The  potential  of  a  medium  for  transfer  of  knowledge  and  skills  has  more  to  do 
with  how  the  information  is  packaged  and  accessed  by  the  learner  than  it  does  with  the 
characteristics  of  the  medium  itself  (Johnston,  1987). 

Information  encoded  as  print  differs  very  little  whether  it  is  presented  by  video,  CRT  or  in  a 
handout.  The  spoken  word  may  be  carried  by  the  instructor’s  voice  in  person,  on  audio  tape  or  by 
video.  The  assumption  is  made  that  learning  does  occur  with  electronic  media  and  that  the  way 
the  information  is  packaged,  accessed  and  presented  can  have  great  effect  on  the  learning 
process.  Given  this  assumption,  the  problem  becomes  how  to  provide  the  tools  and  the  expertise 
to  permit  the  instructional  developer  to  have  choices  in  the  selection  of  an  appropriate  medium 
and  the  most  effective  packaging  of  that  instruction  for  optimal  learning.  An  important 
consideration  in  the  design  and  development  process  is  the  "mindware,"  a  term  coined  by 
Salomon  (1985)  that  refers  to  the  mindset  a  learner  brings  to  the  instructional  process.  It 
includes  the  learner’s  propensities  and  associations  with  media.  Main  (1992)  has  attempted  to 
systematize  these  motivational  factors  by  generating  a  model  of  instructional  design  which 
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integrates  the  affective  domain  into  the  curriculum  development  process.  Although  empirical 
data  is  sparse,  the  attractiveness  of  electronic  media  is  evident  in  the  amount  of  leisure  time  spent 
with  television  and  electronic  games  by  both  children  and  adults.  Though  there  is  some  ongoing 
discussion  in  the  literature  over  how  much  and  what  types  of  media  are  appropriate  to  various 
tasks  (see  Friedman,  Poison  &  Spector,  1991),  we  will  not  attempt  to  explore  these  issues  here 
since  they  are  more  the  province  of  an  ADDA,  SME’s  and  IDD’s  than  they  are  dependent  on  a 
CIDE. 

The  very  strength  of  a  collaborative  instructional  development  environment  is  the  versatility 
to  produce  any  and  all  types  of  media  as  deemed  suitable  to  a  particular  task  by  the  cadre  of 
professionals  employed  for  ISD.  Whether  the  media  need  is  a  published  document,  graphic 
slide,  computer-based  animation,  or  edited  videotape,  the  personal  computer  has  matured 
sufficiently  as  a  platform  to  produce  professional  quality  products.  We  state  here  that  the  PC  is 
sufficiently  robust  to  simultaneously  support  a  panoply  of  media  production  tools,  an  ADDA,  and 
a  collaborative  network  to  form  electronic  work  groups. 

Electronic  work  groups  as  discussed  here  are  the  result  of  computer-mediated  communication 
(CMC).  Analog  communication  networks  are  not  considered  because  of  their  relatively  high  cost 
when  used  for  media  exchange.  Computer-based  communication  systems  range  from  simple 
electronic  mail  to  voice  mail,  interactive  chat  forums,  desktop  video  conferencing,  document 
transfer  and  shared  screen  editing.  Basic  CMC  systems  have  gained  enormous  popularity  over 
the  last  10  years  as  exemplified  by  the  rise  in  usage  of  public  network  services  such  as  MCI 
Mail,  Prodigy  and  America  On-Line,  and  the  worldwide  research  collaboration  taking  place  on 
InterNet,  NSFnet  and  Bitnet.  For  millions  of  people,  checking  their  e-mail  messages  or  logging 
into  an  interactive  chat  conference  has  become  a  daily  ritual.  InterNet,  a  consortium  of 
universities  and  research  institutions,  expects  to  have  more  than  2  million  participants  by  1995 
{Communications  Week,  July  6,  1992,  p.  1). 

These  services  are  active  collaborative  communities.  Over  one  thousand  special  interest 
forums  on  the  InterNet,  for  example,  allow  asynchronous  discussion  of  social,  technological  and 
scholarly  issues  ranging  from  animal  psychology  to  quantum  physics.  The  participants  represent 
a  huge  knowledge  base.  By  actively  sharing  information  they  multiply  their  individual  skills  and 
abilities. 

IS  ELECTRONIC  COLLABORATION  EFFECTIVE? 

The  literature  on  collaborative  work  is  rich  with  examples  of  increased  employee 
productivity.  Though  one  study  of  Air  Force  cadets  found  that  highly  competitive  people 
perform  best  when  given  individual  rewards  (Porter,  Bird,  &  Wunder,  1990),  the  majority  of 
researchers  have  reported  improvement  in  performance  on  complex  tasks  by  collaborative  groups 
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(Bassin,  1988;  Blaye,  Light,  Joiner,  &  Sheldon  1991;  Johnson,  Maruyama,  Johnson,  Nelson  & 
Skon,  1981;  Katz,  Kochan  &  Weber,  1985).  In  their  excellent  literature  review,  Tjosvold  and 
Tsao  (1989)  state: 

Considerable  research,  including  field  experiments,  indicate  that  people  in 
cooperation  compared  to  those  in  competition  exchange  resources,  assist  each 
other,  and  manage  conflicts  constructively  so  that  they  are  all  successful  (p.  189). 

Citing  the  findings  of  Johnson  et  al.  (1981)  they  continue,  "as  they  work  cooperatively, 
employees  explore  issues  and  make  successful  decisions,  and  are  more  productive  especially  on 
complex  tasks  that  benefit  from  sharing  information"  (p.  189). 

According  to  Bassin  (1988),  "It’s  not  the  gifted  individuals  who  make  peak  performance 
possible  as  much  as  the  dynamics  of  belief,  collaboration  and  support"  (p.  64).  Bassin  believes 
cooperative  work  groups  are  effective  because  of  the  resources  of  individual  members,  diversity 
of  ideas,  emotional  support,  mutual  motivation  and  increased  job  satisfaction.  He  feels  that 
isolated  employees  are  at  a  fundamental  disadvantage,  unable  to  grasp  how  their  work  output  fits 
into  the  overall  performance  of  the  organization.  Collaborative  teams  solve  these  problems. 
According  to  Tjosvold  et  al.  (1989): 

In  cooperation,  people  believe  their  goals  are  positively  linked;  one’s  goal 
attainment  helps  others  reach  their  goals.  Alternatively,  mistrust,  individual  tasks, 
and  win/lose  rewards  induce  competition.  Competitors  believe  their  goals  are 
negatively  correlated  so  that  one’s  goal  attainment  makes  it  more  difficult  for 
others  to  attain  their  goals  (p.  189). 

Finally,  members  of  cooperative  work  groups  report  increased  job  satisfaction  and 
organizational  loyalty  (Andrews  and  Jones,  1983;  Bassin,  1988;  Finholt,  Sproull  &  Kiesler, 

1990;  Sproull  &  Kiesler,  1986;  Tjosvold,  Johnson  et  al.,  1981;  Tjosvold  et  al.,  1989).  A  reduced 
sense  of  isolation,  greater  understanding  of  organizational  objectives,  emotional  support  and 
social  interaction  all  seem  to  play  important  roles.  Sproull  and  Keisler  (1986)  point  out  that 
cooperative  work  groups  often  use  electronic  mail  to  provide  a  productive  outlet  for  natural 
desires  for  sociability  and  organizational  attachment.  "People  like  to  be  sociable  at  work.  A 
technology  that  makes  it  easy  to  be  sociable— be  it  a  water  fountain,  coffee  pot,  telephone,  or 
EMS  [electronic  messaging  system]— will  be  used  for  sociability"  (p.  1151).  Likewise,  Tjosvold, 
et  al.  (1983)  suggest  that  cooperative  interaction  strengthens  morale,  commitment  to  the 
organization  and  productivity.  The  positive  experiences  of  working  together  lead  employees  to 
believe  they  have  gained  a  great  deal  from  the  employer;  and  teamwork  binds  them  to  each  other 
and  to  the  organization. 
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It  is  generally  felt  that  members  of  a  cooperative  work  group  benefit  from  the  strengths  of  the 
talented  individuals  of  whom  it  is  comprised  (Bassin,  1988).  Technical  expertise  and  design 
experience  more  readily  cross  the  organizational  lines  in  an  ad  hoc  cooperative  group,  providing 
just-in-time  support  for  mission-critical  objectives  (Finholt,  et  al.,  1990).  Employees  participate 
in  organizational  goals  and  enjoy  increased  productivity  and  greater  job  satisfaction.  But 
traditional  methods  of  forming  and  maintaining  ad  hoc  work  groups  such  as  face-to-face 
meetings  may  cost  an  organization  a  great  deal  in  terms  of  travel,  time  to  distribute  materials,  and 
time  required  to  meet  and  to  schedule  more  meetings,  especially  amongst  geographically  remote 
participants  (Finholt  et  al.,  1990). 

The  literature  provides  sufficient  evidence  that  electronic  collaboration  is  effective.  There  is 
a  note  of  caution.  Changing  communication  patterns  and  protocols  also  changes  organizational 
culture.  These  issues  are  not  addressed  in  this  paper  but  considerable  empirical  evidence  is 
available  and  should  be  examined  before  establishing  capabilities  (see  for  example  Dubrovsky, 
Kiesler  &  Sethna,  1991;  Lea  &  Spears,  1991;  Smilowitz,  Compton  &  Flint,  1989;  Sproull  et  al., 
1986). 

WHAT  ARE  THE  INSTRUCTIONAL  MEDIA  NEEDS  OF  AIR  FORCE  IDD’S 
AND  HOW  ARE  THEY  NOW  BEING  MET? 

A  Delphi  group  of  expert  Air  Force  IDD’s  was  used  to  determine  the  instructional  media 
needs  of  the  Air  Force  and  what  tools  would  most  enrich  the  collaborative  instmctional 
development  environment. 

The  methodology  for  this  study  involved  a  review  of  state-of-the-art  technologies  in  the  field 
of  personal  computers  and  desktop  workstations,  media  production  tools  and  communication 
software.  The  evaluation  was  limited  to  commercial-off-the-shelf  applications  or  products  being 
beta  tested  for  commercial  release.  An  examination  of  the  trade  publications  in  personal 
computing,  desktop  publishing,  digital  photography,  graphic  design,  video  production  and 
communication  networking  was  used  to  establish  a  taxonomy  of  available  products  and  services 
that  appeared  to  have  value  for  instmctional  development.  A  trip  was  made  to  the  InfoMart  in 
Dallas  to  see  some  of  the  candidate  technologies  demonstrated. 

The  information  from  the  technology  review  was  used  to  generate  a  list  of  42  product 
categories  divided  into  three  areas  of  instmctional  design/development:  1)  Instmctional  materials 
development,  2)  Management,  and  3)  Collaboration.  To  provide  a  rational  method  for  evaluating 
the  importance  of  the  functions  represented  by  these  product  categories,  a  combination  of  the 
Delphi  methodology  and  the  Kepner-Tregoe  rational  decision  model  (1965)  was  used. 
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The  Kepner-Tregoe  rational  decision  model  uses  the  technique  of  determining  what  are  the 
essential  outcomes  and  what  are  the  desirable  outcomes  for  any  decision  situation.  It  is  widely 
used  in  the  evaluation  of  competing  systems  because  it  provides  a  quantifiable  method  for 
comparing  products  with  a  variety  of  disparate  features.  The  "must"  category  of  features  must  be 
met  by  all  candidate  systems  or  they  are  dropped  from  further  consideration.  Those  features 
deemed  desirable  but  not  essential  are  labeled  "wants"  and  are  assigned  weights  (usually  by  a 
panel  of  users).  The  evaluation  is  made  by  experts  who  test  the  system’s  ability  on  each  item. 

For  pur  panel  of  experts  we  elected  to  use  Air  Force  IDD’s  from  a  number  of  organizations 
that  would  reflect  a  variety  of  instructional  development  needs  from  standup  courses  to 
computer-based  technical  training.  We  opted  for  this  approach  over  using  consultants  or 
academics  because  of  the  importance  of  user  involvement  in  system  design. 

Systems  development  theory  (Boar,  1984;  Conner,  1985;  Kling,  1991)  and  practical  field 
experience  (Kyng,  1991;  Perin,  1991)  both  indicate  that  potential  users  of  a  system  must  be 
involved  during  the  early  stages  of  design.  Kyng  (1991)  advocates  a  doctrine  of  "mutual 
learning"  where  designers  teach  users  about  the  technological  possibilities  while  users  instruct 
designers  in  the  task  specifics  of  their  work.  Perin  (1991)  discusses  the  problems  created  when 
systems  are  mandated  for  unwilling  users.  Computer  systems  that  extend  the  abilities  of 
subordinates,  and  especially  those  that  may  create  informal  social  fields  among  them,  may 
threaten  managers.  "The  challenge  is  to  create  computer  support  that  acknowledges,  if  not 
incorporates  these  realities,  rather  than  presuming  the  technology  will  by  itself  reform  or 
obliterate  them"  (p.  81). 

Therefore,  while  expert  consultants  might  easily  specify  an  extremely  competent  design 
system  in  terms  of  the  prevalent  ISD  models  and  perceived  needs  of  IDD’s,  there  is  no  certainty 
that  such  a  system  will  be  readily  adopted  by  Air  Force  IDD’s.  For  these  reasons,  we  assembled 
a  Delphi  panel  of  experienced  users  to  assist  in  developing  the  functional  design  requirements  for 
the  CIDE.  We  were  assisted  in  identifying  expert  IDD’s  by  Lt.  Sheila  Robinson  (HQ 
ATC/TTDD)  and  by  Maj.  Richard  O’Neal  (HQ  ATC/XPCR)  at  Randolph  Air  Force  Base. 

Delphi  is  a  technique  developed  by  the  RAND  Corporation  to  be  used  in  technical 
forecasting  or  to  achieve  consensus  among  a  group  of  experts  without  undue  influence  (halo 
effect)  by  prestigious  individuals  (Tersine  &  Riggs,  1976).  For  this  study,  a  Delphi  group  of  10 
experienced  Air  Force  IDD’s  was  selected.  Their  combined  experience  totals  108  years  in 
curriculum  development.  The  participants  are  expert  practitioners  rather  than  a  representative 
sample  of  Air  Force  instructional  developers.  A  survey  by  Walsh,  Yee,  Grozier,  Gibson  and 
Young  (1992)  of  256  Air  Force  personnel  involved  in  developing  computer-based  instruction 
(CBI)  found  the  average  experience  of  the  IDD’s  to  be  just  20  months.  Participants  were 
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selected  for  this  panel  because  of  their  knowledge  and  experience  with  Air  Force  instructional 
design  and  development,  not  because  they  represented  typical  IDD’s. 

Eight  members  of  the  panel  of  experts  were  male  and  two  were  female.  Seven  were  civilian 
employees  of  the  Air  Force  and  three  were  career  military  personnel.  The  level  of  sophistication 
of  the  group  was  quite  high.  One  participant  was  a  manager  of  an  instructional  development 
group.  Although  not  involved  in  the  actual  design  of  instruction  at  this  time,  he  had  more  than  7 
years  of  prior  experience  in  training  development.  He  was  included  because  his  managerial 
responsibilities  included  the  design  and  development  of  all  types  of  instruction  from  traditional 
classroom  to  CBI.  Eight  of  the  members  were  experienced  in  designing  and  developing  CBI  and 
three  of  them  did  this  exclusively.  Five  panel  members  were  involved  with  the  design  of  standup 
training  using  static  media  aids  and  six  develop  dynamic  media  for  their  instructional  programs. 
Seven  of  the  designers  had  at  least  some  experience  in  multimedia  CBI  development.  This  level 
of  expertise  and  experience  in  the  field  made  this  group  well  qualified  to  offer  expert  evaluations 
concerning  the  functional  requirements  and  desired  features  for  a  collaborative  workstation  to 
improve  both  the  productivity  and  quality  of  Air  Force  instructional  design  and  development. 

INSTRUMENTATION  AND  EVALUATION 

A  structured  questionnaire  was  distributed  to  the  panelists  in  which  they  were  asked  to 
specify  the  percentages  of  instruction  created  using  different  types  of  media  both  within  their 
organization  and  Air  Force-wide.  We  then  asked  them  to  tell  us  how  much  instruction  using 
each  type  of  media  they  thought  would  be  most  appropriate  for  use  by  their  organization  and  the 
Air  Force. 

They  were  presented  with  the  list  of  42  candidate  technologies  developed  from  the  trade 
journals,  literature  review  and  vendor  presentations.  They  were  told  their  expertise  was  being 
solicited  to  assist  in  determining  the  design  features  of  a  collaborative  instructional  development 
workstation.  They  were  asked  to  evaluate  each  technology  category  to  determine  if  they  felt  it 
was  essential  (a  "must")  to  quality  curriculum  development.  If  the  technology  was  judged  not  to 
be  essential,  the  panelists  were  asked  to  place  a  value  on  its  worth  (0=valueless  to  20=nearly 
essential)  to  an  instructional  developer. 

Finally,  they  were  asked  about  media  they  can  and  cannot  presently  develop  in-house  and 
their  collaborative  relationships  with  other  designers  and  subject  matter  experts. 

We  sought  to  answer  five  basic  questions  that  bear  directly  on  the  functional  specifications  of 
the  CIDE: 

•  What  types  of  instructional  media  are  presently  being  developed? 
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•  What  types  of  instructional  media  would  IDD’s  prefer  to  develop  if  they  had  more 
resources? 

•  By  whom  is  various  media  now  developed  (IDD’s,  non-training  agencies, 
contractors)? 

•  What  kind  of  collaboration  is  necessary  to  the  development  of  effective  media  for 
instruction? 

•  What  technologies  are  perceived  as  essential  to  IDD’s;  which  are  desirable,  and  which 
are  unnecessary? 

The  data  gathered  was  averaged  and  used  to  rank  order  potential  technologies  that  could  be 
included  in  the  CIDE.  Using  the  Kepner-Tregoe  (1965)  decisioning  system,  we  were  able  to 
determine  which  technologies  constitute  the  necessities  of  the  system  and  which  the  niceties. 
Using  this  "rational"  decisioning  system  helps  forestall  the  desire  to  add  every  available 
technology  under  the  assumption  that  if  we  provide  it  to  designers  they  will  learn  to  want  it  and 
use  it—  the  "Field  of  Dreams"  approach. 

THE  TECHNOLOGIES 

Our  only  constraint  (self-imposed)  was  that  all  software  and  hardware  technologies  specified 
for  the  design  of  the  CIDE  should  be  compatible  with  the  Air  Force  Desktop  III  and  IV.  Our 
intent  in  this  was  not  only  to  reduce  eventual  development  costs  and  to  work  with  a  computing 
platform  that  has  already  been  approved  and  implemented  by  the  Air  Force,  but  also  to  ensure 
compatibility  with  the  Advanced  Instructional  Design  Advisor  being  developed  for  Air  Force 
ISD  (Hickey,  SpectorA  Muraida,  1992). 

Technological  feasibility  was  determined  through  review  of  computer  trade  publications  and 
an  on-site  visit  to  Dallas’  InfoMart.  While  specific  software  and  hardware  selections  will  require 
further  study  and  additional  input  from  potential  users,  there  will  be  a  discussion  below  of 
critical  technologies  that  match  the  user  requirements  determined  by  the  Delphi  panel.  The 
research  and  development  paradigm  is  to  establish  a  rational  ordering  of  functional  requirements 
and  assess  the  status  of  commercial-off-the-shelf  (COTS)  tools  available  to  meet  those 
requirements. 

FINDINGS 

Our  hypothesis  is  that  Air  Force  IDD’s  are  probably  designing  more  standup  instruction  and 
more  instruction  with  static  media  than  they  would  prefer.  If  true,  we  suspect  it  was  because  of 
an  inability  to  design  more  dynamic  curriculum  materials  stemming  from  a  variety  of  reasons 
ranging  from  lack  of  skills  to  lack  of  equipment  to  insufficient  time.  To  test  this  theory,  we 
asked  the  Delphi  panel  for  their  best  estimate  of  the  quantities  of  instructional  media  of  various 
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types  being  produced  by  them,  their  organizations,  and  their  best  estimate  of  the  media  types 
used  Air  Force-wide.  Summaries  of  their  responses  are  contained  in  Figures  1-8. 

As  we  postulated,  individual  Air  Force  IDD’s  generally  feel  they  are  developing  more 
instruction  without  media,  or  instruction  that  is  dominated  by  static  media  than  they  would  prefer 
(Figure  1).  The  mean  portion  of  curriculum  hours  developed  as  standup  instruction  with  no 
media  was  estimated  by  our  panel  to  be  40  percent  within  their  own  organizations  and  32  percent 
overall  for  the  Air  Force.  They  believed  a  more  suitable  amount  of  this  type  of  instruction  would 
be  about  25  percent. 

The  participants  also  indicated  they  would  prefer  to  see  less  instruction  supported  by  static 
media  such  as  slides,  overhead  transparencies,  etc.  (Figure  2).  They  estimated  instruction  with 
static  media  accounted  for  almost  50  percent  of  the  hours  of  instruction  produced  in  their 
organization  and  nearly  60  percent  Air  Force-wide.  Their  preference  was  that  approximately 
one-third  of  the  instructional  hours  be  standup  instruction  supported  by  static  media. 


Standup  Instruction  with  No  Media 
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Participant’s  Organization  (  Air  Force  Wide  (Estimate) 

Figure  1 


Standup  Instruction  with  Static 
Media  (Slides,  Photos,  etc.) 


Participant’s  Organization  1  Air  Force  Wide  (Estimate) 

Figure  2 


The  use  of  dynamic  media  for  instruction  shows  an  opposite  result,  i.e.,  the  participants 
would  like  to  use  dynamic  media  more  than  it  is  being  used  now  (Figure  3).  Participants  would 
like  to  increase  their  organization’s  use  of  dynamic  media  from  30  to  nearly  40  percent  of 
instruction  designed,  and  would  like  to  see  it  account  for  one-third  of  total  Air  Force  instruction. 
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Standup  Instruction  with  Dynamic 
Media  (Slide-Tape,  Video  etc.) 
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Figure  4 


As  we  suspected,  print-based  media  is  still  the  most  widely  used  medium  (Figure  4).  More 
than  90  percent  of  instructional  hours  are  supported  by  some  printed  materials  in  the  form  of 
student  handouts  and  workbooks.  The  effective  penetration  of  desktop  publishing  and  familiarity 
of  nearly  all  instructional  designers  with  paper-based  production  certainly  facilitates  its  ubiquity. 
Nevertheless,  panelists  felt  that  the  amount  could  be  reduced  somewhat  without  damage  to  the 
instructional  process. 


Curriculum  Presented  as  Computer- 
Based  Instruction 
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Figure  5 


Computer-Based  Instruction  with 
Multimedia  Presentation 


p  100 

e  D  90 
^  e  80 


Actual  Preferred  Actual  Preferred 
Participanrs  Organization  1  Air  Force  Wide  (Estimate) 

Figure  6 


While  the  percentage  of  instruction  presently  developed  for  computer-based  delivery  is  low 
(18  percent  within  our  panelist’s  organizations  and  less  than  10  percent  estimated  Air  Force¬ 
wide),  most  participants  would  like  to  see  CBI  use  increased  greatly  (Figure  5).  Although  one 
panelist  charged  exclusively  with  CBI  development  feels  that  CBI  should  only  account  for  10 
percent  of  all  instruction,  other  participants  felt  the  amount  of  CBI  desired  should  be  nearly  one- 
third  of  Air  Force-wide  instruction. 
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Every  respondent  wants  to  see  more  multimedia  CBI  developed  for  Air  Force  instruction 
(Figure  6).  At  present,  one-third  of  ail  CBI  being  developed  within  participants’  organizations 
consists  of  dynamic  multimedia,  along  with  an  estimated  25  percent  of  such  CBI  Air  Force-wide. 

But  panelists  believe  that  the  amount  should  be  pushed  above  50  percent,  that  is,  more  than 
half  of  all  computer-based  instruction  should  be  multimedia.  This  suggests  a  clear  need  for 
interactive  dynamic  multimedia.  However,  as  participant  comments  allude,  many  of  the  tools 
required  to  develop  motivational  multimedia  are  presently  unavailable  in  the  field. 

Air  Force  IDD’s  also  indicate  an  interest  in  developing  more  instruction  for  distance  learning 
applications  (Figure  7).  By  their  estimate  a  scant  two  percent  of  Air  Force  instruction  now 
constitutes  distance  learning.  However,  they  believe  as  much  as  27  percent  of  their  instruction 
has  distance  learning  applications,  and  see  a  potential  for  22  percent  of  total  Air  Force  instruction 
to  be  delivered  remotely. 

Finally,  panelists  felt  that  too  great  an  emphasis  is  now  placed  on  objective  examinations. 
They  show  a  clear  preference  for  developing  performance-based  evaluations  both  within  their 
own  organizations  and  Air  Force- wide  (Figure  8).  The  data  suggests  most  of  the  experts  want 
performance-based  evaluations  to  supplement  rather  than  totally  replace  objective  exams.  Given 
the  nature  of  many  of  the  tasks  for  which  Air  Force  IDD’s  develop  instructional  systems,  it 
seems  likely  that  objective  measures  may  often  be  insufficient.  Many  skills-based  tasks  can  only 
be  effectively  evaluated  by  proficient  performance.  Objective  exams  are  generally  easier  to 
develop,  administer  and  evaluate  than  performance-based  tests,  suggesting  that  IDD’s  may 
benefit  from  tools  that  help  them  develop  more  innovative  evaluation  measures. 
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In  summary,  the  experienced  Delphi  panel  would  like  to  develop  less  standup  instruction  that 
is  unsupported  by  media  or  has  only  static  media.  Although  they  would  like  to  reduce  their 
reliance  on  traditional  text-oriented  student  handouts  and  workbooks,  they  still  want  print 
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support  for  three-fourths  of  their  instruction.  They  would  like  to  increase  the  use  of  dynamic 
media  as  a  support  for  standup  instruction  and  they  would  like  to  increase  CBI  and  in  particular 
the  use  of  CBI  that  includes  multimedia  presentation.  They  would  also  like  to  increase  the 
amount  of  courses  offered  through  distance  learning  systems  and  they  would  like  to  implement 
more  performance-based  evaluations. 

MEDIA  PRODUCTION  METHODS 

Most  of  our  Delphi  panelists  (seven  of  nine  responding)  contract  the  final  development  of 
graphics.  Reasons  cited  for  this  range  from  not  violating  a  base-negotiated  contract  by 
developing  graphics  in-house,  to  a  respondent  who  cites  excessive  time  and  effort  spent  to 
produce  non-professional  looking  graphics.  Only  two  of  nine  IDD’s  report  they  develop  their 
own  final  graphics.  Two  report  they  are  starting  to  develop  more  internally.  One  of  those  cited 
slow  turnaround  by  contractors. 

All  respondents’  organizations  contract  for  printing  services,  though  six  of  nine  provide 
camera-ready  copy,  indicating  the  penetration  of  desktop  publishing.  One  panelist  reports  that 
their  organization  is  beginning  to  desktop  publish  and  hope  to  soon  provide  camera-ready  copy; 
one  panelist  reports  that  contractor  turnaround  is,  "not  very  fast."  Two  panelists  report  that 
printing  services  are  handled  by  contractors,  but  do  not  specify  who  provides  camera-ready  copy. 

Five  of  seven  respondents  use  on-base  contractors  for  all  photography.  Two  of  seven  provide 
their  own  photos.  Similarly,  five  of  nine  respondents  use  on-base  contractors  for  all  video 
footage,  while  two  of  nine  produce  their  own  internally.  Of  these  two  respondents,  one  is  tasked 
with  CBI  development,  the  other  is  a  manager  whose  organization  develops  primarily  textual 
media  and  team  training.  Two  of  nine  share  the  task  of  video  development  with  contractors. 
Three  of  seven  respondents  contract  for  slide/tape  program  production,  two  produce  in-house  and 
two  report  no  slide/tape  productions.  Three  of  eight  panelists  report  audio  production  is  provided 
by  on-base  contract  while  two  develop  in-house.  Three  panelists  state  audio  production  services 
are  not  available  (even  though  computer-based  audio  production  tools  are  highly  developed  and 
inexpensive). 

ESSENTIAL  AND  DESIRED  DEVELOPMENT  TOOLS 

Respondents  were  asked  to  determine  with  a  yes  or  no  vote  whether  a  variety  of  computer-based 
media  development  tools  were  essential  to  ISD.  Where  they  voted  no,  they  were  asked  to 
determine  the  usefulness  of  the  tool  for  ISD  on  a  scale  of  0-20.  Each  "yes"  vote  is  valued  at  30 
points,  while  each  "no"  vote  is  valued  at  its  given  weight.  These  data  are  totaled,  divided  by  30 
and  used  to  create  the  Kepner-Tregoe  decision  tree  shown  in  Figure  9.  Tools  that  receive  60 
percent  of  possible  points  (180  points  of  the  300  points  possible)  are  considered  to  be  essential  to 
ISD  and,  therefore,  to  the  CIDE  workstation;  those  totaling  50  percent  (or  150  points)  are 
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Test  Development 
Flow  Charting 
Budgeting 
Monochrome  Printer 


lesktop  Video  Conferencing 
3-D  Graphics 
3-D  Scanner 
Laserdisk  Development 
Chat  Conferencing 


deemed  highly  desirable;  others  below  50  percent  are  considered  useful  in  proportion  to  their 
weights.  Note  that  no  tool  received  a  weight  below  30  percent  (90  points),  and  at  least  two 
panelists  considered  any  given  tool  absolutely  essential  (yes  votes).  Thus,  all  of  these  tools 
should  probably  be  available  as  add-on  features  to  the  workstation  to  support  the  task  needs  of 
particular  designers. 

Clearly  Air  Force  instructional  designers  and  developers  would  like  to  be  producing  more 
dynamic  and  motivational  media  for  standup  instruction,  computer-based  instruction  and  for 
distance  learning  applications.  A  variety  of  reasons  for  the  present  lack  of  media  are  suggested 
in  the  panelists’  comments  on  their  organizations’  current  arrangements  for  final  media 
production. 

One  respondent  states  that  the  existing  base-negotiated  graphics  contract  legally  prevents 
them  from  developing  graphics  in-house.  One  is  simply  lacking  sufficient  equipment.  Three 
others  cite  training  and  poor  final  quality  of  in-house  work  due  to  "seldom  used  but  technically 
difficult  skills."  Four  of  the  respondents  bemoan  long  turnaround  time  for  most  contracted 
media,  while  three  others  cite  low  quality  in  contractor-developed  media  due  to  poor 
communication  or  insufficient  familiarity  with  the  subject  matter.  Despite  these  problems,  all 
parties  indicated  the  need  to  use  more  dynamic  and  motivational  instructional  media. 

The  literature  cited  earher  suggests  that  many  of  these  problems  could  be  remedied  by  the 
installation  of  a  collaborative  network.  Creativity  and  expertise  hurdles  can  be  surmounted  by 
special  interest  groups  and  just-in-time  technical  support.  Communications  problems  with 
contractors  can  be  circumvented  with  more  timely  collaborative  sessions.  With  the  appropriate 
tools,  more  preparatory  development  work  can  take  place  in-house  even  if  prior  agreements 
stipulate  that  contractors  must  produce  final  media.  And  with  the  right  tools,  designers  will  have 
the  freedom  to  explore  more  creative,  dynamic  and  motivational  media  solutions  to  instructional 
problems. 

From  the  Kepner-Tregoe  decisioning  tree  it  is  fairly  obvious  that  IDD’s  themselves  recognize 
this.  Of  course  there  was  unanimous  agreement  for  word  processors,  author  ware,  flowcharts  and 
test  development  tools—  the  staples  of  the  trade.  But  not  surprisingly,  there  was  extremely  strong 
interest  in  desktop  publishing,  two-dimensional  graphics,  image  scanners,  simulations 
development,  digital  and  analog  video  editing,  video  and  photography  transfer,  and  CD-ROM 
input  and  publishing.  These  are  tools  to  create  a  media-rich  instructional  environment.  They  are 
not  simple  to  use,  requiring  technical  proficiency  and  creativity,  but  IDD’s  understand  their  value 
to  the  development  of  highly  motivational,  dynamic  media. 
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The  data  indicate,  as  well,  that  they  understand  the  value  of  collaboration  with  peers  and 
specialists  to  quality  ISD.  Every  respondent  reported  routine  collaboration  with  a  subject  matter 
expert.  Four  participants  agreed  with  the  panelist  who  stated,  "[it  is]... impossible  to  develop  any 
kind  of  quality  training  without  an  SME."  In  addition,  four  of  nine  cite  frequent  collaboration 
with  other  IDD’s  for  ideas  and  evaluation;  five  of  nine  report  collaboration  with  graphics 
specialists;  and  three  of  nine  report  contact  with  technical  experts  and  media  specialists:  "without 
this  link,  our  product  would  not  get  out." 

Presently,  the  bulk  of  this  collaboration  is  conducted  face-to-face  or  over  the  telephone. 
Where  media  production  and  graphic  design  specialists  are  concerned,  considerably  more  work  is 
conducted  face-to-face  than  by  any  other  method.  Yet,  most  of  these  experts  cite  time  and 
communication  factors  as  primary  impediments  to  more  extensive  use  of  motivational  media. 
Clearly  a  well-implemented  collaborative  network  that  connects  IDD’s,  SME’s  and  media 
production  specialists— including  contractors— would  solve  most  of  the  aforementioned  problems, 
streamline  collaborative  processes  by  eliminating  many  face-to-face  meetings,  and  encourage  the 
creative  development  of  more  effective,  dynamic  and  motivational  media. 

Finally,  the  experience  of  the  Navy  cited  by  Cantor  (1988)  suggests  that  centralized  archival 
of  reusable,  adaptable  media  would  save  time  and  money,  while  encouraging  IDD’s  to  make  use 
of  the  best  stock  media  aveiilable.  This  type  of  networked  archive  would  make  media  produced 
by  the  most  highly  skilled  and  talented  producers  readily  available  to  IDD’s  throughout  the  Air 
Force’s  widely  distributed  ISD  agencies. 

CAN  A  COLJ  .ABORATIVE  INSTRUCTIONAL  DESIGN  ENVIRONMENT  BE  CREATED 
WITH  CURRENT  COMMERCIAL-OFF-THE-SHELF  TECHNOLOGY? 

The  answer  to  this  questions  is  a  qualified  yes.  Every  item  on  the  list  of  42  functions 
identified  as  essential  or  desirable  for  at  least  some  IDD’s  is  available  right  now.  The  open 
architecture  and  communication  networking  capabilities  are  present.  What  cannot  be  answered 
by  this  study  is  what  integration  software,  degree  of  data  interchange  standards  and 
communication  data  speeds  are  necessary  and  available  for  implementation  of  every  function. 

To  answer  this  question  definitively  will  require  additional  study,  prototype  development  and 
beta  testing  which  is  strongly  recommended.  A  beta  test  would  create  a  field  laboratory 
environment  that  could  be  useful  in  answering  a  variety  of  research  questions  regarding  process 
and  task  procedures  for  optimal  use  of  the  collaborative  instructional  development  environment. 
Therefore,  the  constraining  factors  for  such  a  system  require  further  research  and  are  more  related 
more  to  software  and  user  interface  issues  than  to  hardware  and  software  capabilities. 
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SUMMARY 

As  indicated  throughout  the  study,  the  mere  provision  of  development  tools  to  IDD’s  does 
not  empower  them  individually  to  design  efficiently  or  effectively.  Instructional  Systems  Design 
(ISD)  is  a  complex  task  requiring  variously  the  input  of  IDD’s,  SME’s,  graphic  designers,  video 
production  specialists,  technical  writers  and  CBI  programmers.  While  increased  attention  to  the 
user  interface  has  created  a  generation  of  media  production  software  much  more  friendly  to  the 
average  user,  true  mastery  of  any  such  tool  is  the  result  of  both  practice  with  the  software  and  a 
thorough  understanding  of  the  knowledge  domain  to  be  represented  and  the  traditions  and 
techniques  native  to  the  media  being  developed.  The  proliferation  of  desktop  publishing 
provides  an  illustrative  example.  While  user-friendly  programs  like  PageMaker  and  Ventura 
Publisher  brought  computer-based  publishing  to  everyone,  they  did  not  communicate  the 
traditions  of  the  typographer’s  art.  As  a  result,  the  average  quality  of  typeset  materials  now 
varies  a  great  deal.  For  expert  typesetters,  the  software  tools  were  a  productivity  boon,  allowing 
them  to  create  high-quality  typeset  pages  more  quickly  and  less  expensively  than  ever  before. 
However,  in  the  hands  of  the  non-typesetter,  they  allowed  only  the  quick  production  of  readable 
but  inelegant  pages  devoid  of  ligatures,  gendered  quotes,  and  properly  kemed  letter  pairs.  To 
quote  Bob  Krejci,  "There  is  no  one-person  authoring  tool  that  can  produce  the  kind  of  product 
that  an  experienced  staff  of  designers,  subject  matter  experts,  artists,  and  programmers  can 
develop...  There  is  no ‘Van  Gogh  in  a  spray  can’ product"  (1992). 

This  is  not  to  suggest  that  there  is  not  merit  in  providing  IDD’s  with  the  wide  variety  of  tools 
required  to  develop  affective  instructional  media.  On  the  contrary,  while  questions  remain  as  to 
the  amount  and  design  of  potent  media  for  various  instructional  objectives  and  within  specific 
domains  of  knowledge,  there  is  no  doubt  that  visual  and  auditory  media  can  be  an  appropriate 
and  highly  effective  means  of  organizing  and  presenting  some  information  (Gildea,  Miller  & 
Wurtenberg,  1990).  Analysis  of  specific  appropriate  forms  and  applications  of  graphics  have 
been  begun  with  respect  to  the  development  of  an  Automated  Instructional  Design  Advisor 
(Friedman,  et  al.,  1991).  While  an  AIDA  may  well  include  guidance  towards  appropriate 
applications  and  designs  for  graphic  instructional  media,  the  goal  of  a  development  workstation 
should  be  to  provide  robust  fully-featured  design  tools.  Whether  a  screw  or  nail  is  appropriate  is 
the  decision  of  the  carpenter  and  architect.  A  good  toolbox  contains  both  driver  and  hammer. 

For  example,  the  graphic  design  and  imaging  segments  of  the  PC  market  have  matured 
sufficiently  that  there  are  a  variety  of  extremely  competent  software  packages  presently  available 
for  the  development  of  fine  art  (e.g..  Fractal  Design  Painter),  line  art  (e.g.,  Corel  Draw,  Adobe 
Illustrator),  CAD  (e.g.,  AutoCad,  Eazy  Cad),  solid  model  rendering  (e.g.,  AutoCad,  Renderman) 
and  photo  image  processing  (e.g.,  Image-In,  Photo  Finish).  While  the  Air  Force  may  choose  to 
standardize  on  one  or  more  of  these  packages,  the  CIDE  workstation  will  be  designed  to 
accommodate  one  or  all  of  them  in  ad  hoc  arrangements  to  support  the  task  at  hand.  However, 
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any  such  integration  of  new  tools  requires  an  equal  commitment  to  technical  and  creative  support 
services  to  aid  in  the  transfer  of  sufficient  expertise  to  make  the  tools  useful.  The  creation  of  ad 
hoc  collaborative  work  groups  can  address  the  problem  of  providing  technical  development 
assistance  to  less  experienced  IDD’s  and  those  without  sufficient  expertise  in  development  of 
specific  types  of  media.  Simultaneously,  collaboration  should  improve  worker  efficiency  and  job 
satisfaction  as  well  as  creating  a  community  of  expertise  and  a  professional  growth  environment 
for  the  motivated  IDD. 

The  superimposition  of  a  collaborative  open  network  on  the  instructional  development 
environment  will  permit  ad  hoc  working  arrangements  for  mentor/apprentice  relationships, 
creative  and  technical  consulting  support  and  multiple  problem  solving  perspectives  for 
individual  IDD’s  as  well  as  bringing  other  benefits  to  improve  both  ISD  and  IDD  performance. 

A  collaborative  open  network  among  instructional  developers  may  include  technologies  as 
commonplace  as  fax,  e-mail  and  voice  mail  or  those  as  esoteric  as  ISDN-based  digital  document 
transfer  and  two-way  video  conferencing.  Fundamentally,  it  constitutes  the  creation  of  open 
communication  channels  that  engender  the  formation  of  ad  hoc  work  groups  and  technical 
interest  forums,  allow  the  transfer  of  documents  and  resource  materials,  and  provide  an  easily 
accessed,  non-threatening  means  to  seek  technical  help  and  creative  assistance. 

Building  an  open  network  will  allow  both  synchronous  (live  chat  forums  or  video 
conferences)  and  asynchronous  (E-mail,  fax)  communications  amongst  IDD’s.  But  perhaps  just 
as  valuable,  it  will  enable  document  transfer  for  evaluation  purposes,  scheduling  and 
coordination,  and  sharing  of  valuable  resources  such  as  adaptable  existing  media  and  a 
centralized  archive  for  some  materials.  Perez  (1992)  in  an  exploration  of  traits  of  the  expert 
training  developer  discovered  that  senior  designers  developing  instruction  through  a  team 
approach,  "...developed  formal  conventions  and  guidelines  to  insure  the  uniform  execution  of  the 
instructional  design"  (p.  13).  CMC  can  support  this  kind  of  control  and  coordination  as  well  as 
increasing  efficiency  in  the  group  design  process.  Cantor  (1988)  reported  that  an  automated 
curriculum  design  environment  developed  for  the  Navy  that  included  archiving  of  boilerplate  text 
and  graphics  as  a  shared  resource  reduced  time  spent  on  repetitive  work  by  allowing 
incorporation  and  adaptation  of  existing  materials.  He  cited  an  aggregate  reduction  in  ISD  time- 
on-task  from  between  45  and  66  percent. 

The  study  by  Walsh  et  al.  (1992)  of  Air  Force  CBT  developers  reinforces  the  value  a 
collaborative  instructional  development  environment  could  provide.  They  found  78  percent  of 
CBI  development  team  members  were  inexperienced  in  CBI  design  and  development.  More  than 
one  in  four  of  the  development  team  members  felt  the  team’s  activities  were  not  well  coordinated 
and  that  communication  between  team  members  was  unclear  and  ineffective. 
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A  formal  job/task  analysis  was  not  performed  in  one  of  every  three  CBI  development  efforts. 
About  one  in  four  CBI  instructional  developers  relied  on  previous  course  materials  or  analyses. 
Forty  percent  used  learning  objectives  modified  from  previous  lessons.  Only  one-half  of 
developers  indicated  a  media  analysis  was  performed  as  part  of  the  CBI  design  process,  and 
when  an  analysis  was  performed  a  subject  matter  expert  was  used  over  two-thirds  of  the  time.  Of 
the  253  CBI  designers  surveyed,  not  one  indicated  a  media  specialist  was  involved  in  the  media 
analysis  process.  Of  development  team  members,  just  over  ten  percent  were  described  as  media 
experts  and  they  were  all  graphic  artists.  For  the  CBI  projects,  95  percent  contained  some 
graphic  components  (icons,  charts,  tables,  diagrams,  maps,  equipment,  human  figures,  even 
animation);  48  percent  had  audio  content  (bells,  beeps,  tunes  as  rewards,  signals,  music,  engines 
and  verbal  commands,  questions,  etc.);  and  45  percent  included  some  still  or  motion  video  (for 
identification  of  equipment,  body  parts,  panels,  etc.,  procedures  and  interpersonal  and 
communication  skills).  The  most  commonly  cited  reasons  for  not  using  multimedia  were  lack  of 
capability,  not  enough  time  and  not  being  trained  for  development.  There  was  no  mention  at  all 
of  interactive  media  applications. 

Walsh’s  survey  of  practicing  computer-b^sed  instructional  developers  strongly  indicates  that 
Air  Force  CBI  development  could  benefit  substantially  from  the  availability  of  a  collaborative 
instructional  development  environment  as  outlined  in  this  study.  The  data  from  the  Delphi  group 
indicates  non-CBI  instructional  development  needs  a  similar  capability. 

Providing  computer-based  media  development  tools  in  a  collaborative  environment  should 
streamline  many  work  processes  and  stimulate  interactive  evaluation  of  instructional 
components.  Tessmer  and  Wedman  (1992)  discovered  that  the  most  common  reason  cited  by 
professional  IDD’s  for  skipping  an  ISD  activity  was  not  lack  of  money  or  experience  but  lack  of 
time.  They  state,  "A  means  of  ‘cutting  comers  while  controlling  risk’  in  ID/D  [ISD]  project[s] 
needs  to  be  developed"  (p.  16).  We  believe  the  CIDE  can  assist  in  cutting  those  comers. 

CONCLUSIONS 

The  Air  Force  should  explore  implementation  of  a  collaborative  instmctional  development 
environment  workstation.  All  the  technologies  involved  are  readily  available  in  COTS  packages. 
Many  of  the  individual  media  development  technologies  such  as  flat  bed  scanning,  graphic 
design,  desktop  publishing,  author  ware  and  laser  printing  are  already  in  place  in  some  ISD 
organizations.  Issues  to  be  resolved  include: 

1)  the  bandwidth  and  best  method  for  ad  hoc  networking  and  multi-network 
management  (e.g.,  simple  ethemet,  ISDN,  FDDI,  ATM,  SNMP,  etc.); 

2)  whether  members  of  the  collaborative  community  can  benefit  from  broadband 
communications  technologies  such  as  desktop  video  teleconferencing,  voice  mail  and 
groupware  (shared  graphic  and  text  editing); 
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3)  what  kinds  of  materials  should  be  centrally  stored  in  a  Multimedia  Education  and 
Training  Archive  (META)  and  whether  those  archives  should  be  maintained  within 
each  ISD  organization  or  by  on-base  visual  information  (VI)  agencies  such  as  Combat 
Camera,  or  both. 

4)  strategies  for  perusing,  indexing  and  previewing  contents  of  the  META,  including 
Boolean  search  techniques  and  indexed  multimedia  information  retrieval. 

5)  determining  any  potential  adverse  consequences  of  installing  a  CMC  network  and 
developing  strategies  to  offset  them. 

With  the  development  of  AIDA,  the  firm  adoption  of  a  standardized  computer  platform,  and 
the  maturation  of  computer-based  media  production  tools  ranging  from  graphic  design  to  desktop 
digital  video  editing,  the  time  is  ripe  for  implementation  of  an  instructional  development 
environment  with  standard  tools,  central  archiving,  and  a  collaborative  network  for  exchange  of 
creative  and  technical  support,  reusable  media,  and  formative  evaluation.  Changes  to 
organizational  culture,  while  predictable,  will  remain  minor  and  manageable.  In  reality  they  will 
most  likely  contribute  to  increased  employee  satisfaction  and  organizational  loyalty.  Without 
question,  such  a  system  will  contribute  to  the  development  of  more  effective,  motivational 
instruction. 
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